A. Field of the Invention
The present invention relates to methods and apparatus for forming in a three-dimensional image space two -dimensional views of an object. More particularly, the invention relates to a method and apparatus for remotely measuring and recording the position and orientation of an ultrasonic imaging transducer while each of a plurality of two-dimensional image slices is obtained of an internal biological feature by the transducer, and assembling in a three-dimensional space accurately oriented scaled and proportioned views of the slices, thereby facilitating three-dimensional visualization of the feature.
B. Description of Background Art
Acquiring and viewing of two-dimensional ultrasound images has long been a useful non-invasive, non-destructive test method which yields valuable information enabling the visualization of otherwise invisible structures, in diverse fields such as medicine and materials inspection. For example, ultrasonic imaging is routinely used to acquire plan-view images of a fetus within the mother""s womb, or of the otherwise invisible honeycomb cell structure in metal panels, so constructed to provide a high rigidity/strength-to-weight ratio. However, a problem exists with existing ultrasonic imaging techniques, particularly when these technologies are used to form images of irregularly shaped objects, including internal biological features (IBF""s) such as a fetus. Thus, a three-dimensional visualization of an IBF oftentimes must be performed in real-time by a doctor or other healthcare professional who is acting as an ultrasonographer while a sequence of ultrasound scans are made on a patient. To form any ultrasonic image or sonogram of an IBF or other such feature, an ultrasonic imaging wand which contains an ultrasonic energy transducer is used. In a transmit mode, the transducer is electrically energized to transmit a fan-shaped scanning beam of ultrasonic energy; in a receive mode, the transducer receives ultrasonic signals reflected from an object and converts the ultrasonic signals to electrical signals which are used to form an image of the object on a monitor screen. The reflected signals received by the transducer are displayed on the screen in a two-dimensional pattern corresponding to the scanned beam of ultrasonic energy emitted by the transducer when the transducer is operated in the transmit mode, the brightness or color of displayed image elements or pixels on the screen being proportional to the strength of the received signals.
To form a three-dimensional visualization of an IBF or other feature of interest, a sequence of two-dimensional views or sonograms are made by varying the orientation and/or location of the ultrasound wand relative to the feature, thus causing the transmitted and received ultrasound beams to xe2x80x9cslicexe2x80x9d the feature at different angles and/or locations. Such xe2x80x9con-the-flyxe2x80x9d visualizations of the three-dimensional shape of a feature, made from a sequence of two-dimensional image slices, is problematic for a number of reasons. For one thing, it requires a substantial degree of skill and experience to perform meaningful visualization. Moreover, the procedure requires that the wand be repositioned or panned continuously in the area of interest for time periods which may be discomforting to a patient. Also, there is no practical way to preserve on-the-fly mental visualizations of an IBF. Therefore, although it is possible to record and preserve individual sonograms, it is usually impractical if not impossible for the ultrasonographer to recreate three-dimensional views of results of an examination at a later date, or to transmit 3-D views to a different healthcare professional for his or her review.
There are existing machines which are capable of tracking the position and orientation of an ultrasonic imaging wand and associating the instantaneous position of the wand with the ultrasound image acquired at that time. However, such machines are extremely expensive and do not afford a capability for retrofitting to existing ultrasound machines.
In U.S. Pat. No. 5,967,979, issued Oct. 19, 1999, the present inventor, Geoffrey L. Taylor, disclosed with Grant D. Derksen a Method And Apparatus For Photogrammetric Assessment Of Biological Tissue. In that patent, a remote wound assessment method and apparatus was disclosed in which an oblique photographic image is made of a surface wound and a target object such as a plate containing a rectangular image and placed near the wound. Using a novel method of determining vanishing points where a photographic image of parallel lines on the target object intersect, coordinate transformations are calculated which map the oblique image of the rectangle into a normal image thereof. Using the same coordinate transformations, an oblique image of a wound adjacent to the target plate is mapped into a normal, i.e., perpendicular view thereof, allowing precise determination of the true size and outline shape of wound features. The ""979 patent also disclosed an enhancement of the novel planar feature mapping method and apparatus with three-dimensional feature mapping. Thus, according to the method, two separate images of a wound and target plate are formed by moving the camera to two different locations which provide two different oblique views from which three-dimensional topographical features of a wound surface may be measured. Although the method and apparatus disclosed in the ""979 patent have proved to be highly successful in evaluating surface features of biological tissue, the problem of conveniently forming three-dimensional views of internal biological features has been heretofore unsolved, motivating the present invention.
An object of the present invention is to provide a method and apparatus for forming from a plurality of two-dimensional image slices of an object a three-dimensional representation of the image slices.
Another object of the invention is to provide a method and apparatus for forming from a plurality of relatively thin image scans which intersect an object at different angles and/or from different vantage points a three-dimensional representation of the image slices, thus facilitating visualization of the object, including heights of various features of the object.
Another object of the invention is to provide a method and apparatus for forming from a plurality of thin image scans which intersect an object at different heights a three-dimensional representation of the image slices, thereby enabling visualization of the object including heights of various features of the object.
Another object of the invention is to provide a method and apparatus for remotely measuring in a three-dimensional coordinate space locations and orientations of a sensor used to gather data.
Another object of the invention is to provide a method and apparatus for remotely measuring the location and orientation of an ultrasonic transducer used to form ultrasound images whereby the location and orientation of features imaged by the transducer may be precisely reconstructed in a three-dimensional coordinate space.
Another object of the invention is to provide a method and apparatus which photogrammetrically monitors a target plate attached to an ultrasonic imaging transducer wand, as the wand is moved relative to an object of interest, and which performs coordinate transformations of a sequence of oblique images of the target plate to thereby map a sequence of relatively thin, quasi two-dimensional ultrasound image scans of an object obtained by the transducer wand at various orientations relative to the object into a sequence of object feature images of correct relative size, shape and location within a three-dimensional coordinate system, from which a three-dimensional visualization of the object is constructed.
Another object of the invention is to provide a method and apparatus for photogrammetrically monitoring ultrasonic image-forming scans of internal biological features, in which a target plate attached to a scanning ultrasonic transducer wand is photographically monitored to thereby determine and record the precise location and orientation of the wand during each of a sequence of scans, coordinate transformations of each oblique wand and target plate image performed to obtain a sequence of normal view images of the target plate, and, using the oblique-to-normal view transformations of target plate images, a sequence of ultrasonically formed scanned images or sonograms are assembled into a composite three-dimensional view from which internal biological features may be visualized.
Various other objects and advantages of the present invention, and its most novel features, will become apparent to those skilled in the art by perusing the accompanying specification, drawings and claims.
It is to be understood that although the invention disclosed herein is fully capable of achieving the objects and providing the advantages described, the characteristics of the invention described herein are merely illustrative of the preferred embodiments. Accordingly, do not intend that the scope of my exclusive rights and privileges in the invention be limited to details of the embodiments described. I do intend that equivalents, adaptations and modifications of the invention reasonably inferable from the description contained herein be included within the scope of the invention as defined by the appended claims.
Briefly stated, the present invention comprehends a method and apparatus for photogrammetrically monitoring the position and orientation coordinates of a sensor being used to acquire a sequence of sensor images of an object, performing a first coordinate transformation to correctly orient the sensor images, and constructing a three-dimensional representation of the correctly oriented sensor images, thereby permitting three-dimensional visualization of the object.
According to the present invention, an optical imaging and recording instrument such as a video camera, camcorder or digital camera is used to form a sequence of photographic images, at arbitrary, typically oblique angles, of a target plate attached to an ultrasonic transducer wand while a sequence of ultrasound image scans is being made of an object of interest, e.g., a fetus within the mother""s womb. During this step, a separate recorded image of the target plate and ultrasound wand is associated with each ultrasound image scan, which is typically a relatively thin, quasi two-dimensional xe2x80x9cslicexe2x80x9d of the object. A sequence of two-dimensional ultrasound image slices is formed by changing the orientation and/or location of the ultrasound wand for each scan, thus obtaining different ultrasound views of the object.
According to the present invention, the target plate has visual features of known dimensions which permit measurement of its distance from, and orientation with respect to a fixed monitoring device such as a video camera which may be temporarily secured to a fixed structure such as a bed on which a patient is lying. For example, the target plate may contain at least one pair of lines that intersect at a known angle, and preferably contains two pairs of parallel lines that are mutually perpendicular, forming a rectangle. When photographed at an arbitrary oblique angle, the image of the target rectangle is in general a quadrilateral. A coordinate transformation and image mapping method is then used to map the intersecting lines of an arbitrary image such as a quadrilateral into the rectangular xe2x80x9creal worldxe2x80x9d shape of the target plate. A preferred method of performing the coordinate transformation and image mapping is that disclosed in U.S. Pat. No. 5,967,979. Using the same coordinate transformation which is used to map an oblique view of the image plate into a normal view thereof, the distance of the wand from the video camera, and its angular orientation with respect to the camera, may be precisely determined for each ultrasound image scan performed by the wand. Also, since the scan pattern of ultrasonic energy emitted by the wand bears a fixed relationship to the wand, precisely determining the position and orientation of the wand precisely determines the location and orientation of each ultrasound image slice relative to a patient and object of interest. The novel method and apparatus according to the present invention utilizes that information to calculate a coordinate transformation matrix which is then used to construct a three-dimensional image representation of the sequence of two-dimensional image slices, utilizing the orientation and position of each slice relative to a fixed reference frame. This three-dimensional image representation of sensor image slices enables an object scanned by the sensor to be visualized in three dimensions.